Equi-Tuning: Group Equivariant Fine-Tuning of Pretrained Models

We introduce equi-tuning, a novel fine-tuning method that transforms (potentially non-equivariant) pretrained models into group equivariant while incurring minimum L_2 loss between the feature representations of and models. Large can be equi-tuned for different groups to satisfy needs various downstream tasks. Equi-tuned benefit from both equivariance as an inductive bias semantic priors provide applications equi-tuning on three tasks: image classification, compositional generalization in language, fairness natural language generation (NLG). also group-theoretic definition NLG. The effectiveness this is shown by testing it against standard empirical experimental results using variety models: Alexnet, Resnet, VGG, Densenet classification; RNNs, GRUs, LSTMs generalization; GPT2 test these benchmark datasets across all considered tasks show generality proposed method.